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Abstract 


A  C4ISR  architecture  for  future  forces  is  a  major  concern  to  the  Army.  This  report 
describes  progress  of  an  on-going  project  to  develop  a  framework  for  assessing  individual 
communication  technologies  and  concepts  by  accounting  for  technological  and  operational 
detail.  Assessments  of  communication  performance  (e.g.,  message  delay  and  message 
completion  rate)  factor  terrain,  mobility,  and  other  scenario  specific  details  via  high-resolution 
simulations.  For  such  measurements,  excessive  run-times  can  be  a  problem.  This  is  usually  the 
case  for  high-resolution  simulation  of  communication  networks.  However,  in  this  paper,  it  is 
shown  that  high-resolution  communication  network  simulation  runs  (using  Qualnet©),  although 
time  consuming,  can  be  used  to  capture  the  dynamics  of  communication  performance  in  closed 
form  expressions  or  meta-models.  The  meta-models  can  then  be  embedded  into  force-on-force 
simulation  (JANUS)  to  get  perhaps  the  most  important  performance  measure,  e.g.,  warfighter 
effectiveness.  This  forms  a  framework  that  supports  detailed,  scenario  specific  examination  of 
the  impact  of  C4ISR  on  warfighter  effectiveness. 


1.  Introduction 


A  C4ISR  architecture  for  future  forces  is  a  major  concern  to  the  Army.  This  report 
describes  progress  of  an  on-going  project  to  develop  a  framework  for  assessing  individual 
communication  technologies  and  concepts  by  accounting  for  technological  and 
operational  detail.  Specifically,  this  paper  reports  on  efforts  to  provide  assessments  by 
accounting  for  terrain,  mobility,  and  other  scenario  specific  details  via  high-resolution 
simulations  and  other  methods. 

There  are  various  types  and  levels  of  measures  of  C4ISR  performance.  At  a 
detailed  level,  communication  performance  measures  like  message  delay  and  message 
completion  rate,  which  are  a  function  of  terrain  and  many  other  technical  details,  can  be 
assessed  with  simulation  tools.  For  such  measures,  excessive  run-times  can  be  a 
problem.  In  general,  high-resolution  simulation  of  communication  networks  is  usually 
expensive  in  this  regard.  In  this  paper,  it  is  shown  that  high-resolution  communication 
network  simulation  runs,  although  time  consuming,  can  be  used  to  capture  the  dynamics 
of  communication  performance  in  closed  form  expressions  or  meta-models.  The  meta¬ 
models  are  then  embedded  into  force-on-force  simulation  (JANUS)  to  get  perhaps  the 
most  important  performance  measure,  e.g.,  warfighter  effectiveness.  This  forms  a 
framework  that  supports  detailed,  scenario  specific  examination  of  the  impact  of  C4ISR 
on  warfighter  effectiveness. 

This  paper  describes  a  framework  for  studying  communication  networking 
capabilities.  An  existing  high  fidelity  communication  network  simulation  tool  called 
Qualnet©  is  used.  Simulation  experiments  are  described  along  with  corresponding 


statistical  analyses  of  the  results.  The  communication  network  meta-models1  developed 
are  intended  for  use  as  part  of  a  larger  effort  to  embed  representations  of  dynamic 
performance  of  communication  network  into  force-on-force  simulations.  A  number  of 
parameters  are  factored  including  antenna  technologies,  frequency  utilization,  UAY  and 
SATCOM  concepts  and  usage,  and  information  distribution  options.  These  are  all 
technology  concepts  and  options  that  affect  communication  and  connectivity  and  hence 
warfighter  effectiveness.  In  general,  this  report  considers  how  communication  models  can 
be  utilized  so  that  the  impacts  of  various  C4ISR  technology  options,  as  well  as  other 
situational  factors,  are  reflected  in  combat  simulations. 

This  report  is  organized  as  follows.  In  the  first  section,  a  description  is  given  of  the 
methods  and  tools  used  to  measure  network  performance  and  factors  of  interest  that 
might  affect  it.  In  section  2,  details  on  the  meta-modeling  approach  utilized  are  presented 
and  a  list  of  the  factors  of  interest  to  this  study  is  provided.  This  is  followed  by  a  section 
on  some  general  network  simulation  runs  that  helped  to  identify  the  trade-offs  associated 
with  communication  and  information  dissemination  options  for  C4ISR  architectures. 
Section  4  describes  the  major  research  effort  of  this  report,  which  is  the  result  of  a  meta¬ 
modeling  effort  to  capture  the  dynamic  communication  performance.  This  is  done  by 
focusing  the  modeling  effort  on  four  distinct  terrain  boxes.  Observations,  conclusions, 
and  a  discussion  of  on-going  and  future  work  are  also  included.  Appendix  A  provides 
more  details  on  synthesized  models. 


1The  concept  of  meta-modeling  is  related  to  response  surface  modeling. 


2.  Methods  and  Tools 


One  goal  of  the  project  documented  in  this  report  is  to  translate  various 
communication  technology  options  into  impacts  on  network  performance.  The  main 
approach  towards  achieving  this  goal  is  summarized  as  follows:  Various  technologies  and 
factors  that  impact  network  performance  are  chosen  and  set  at  levels  of  interest.  Qualnet 
simulations  are  run.  Network  performance  during  these  simulations  is  recorded.  The 
outcomes  of  these  simulation  experiments  are  used  to  construct  the  performance  models 
to  be  used  in  combat  simulations.  This  is  further  described  as  five  steps: 

1 .  Determine  the  important  factors  that  impact  network  performance. 

2.  Use  an  existing  commercial  high  fidelity  communication  network 
simulator  to  gather  data  on  performance  responses. 

3.  Develop  performance  models  (meta-models)  from  the  data. 

4.  Incorporate  functions  into  combat  simulator  to  assess  impact  on 
warfighter. 

5.  Evaluate  functions  to  assess  technology  options  in  terms  of  other 
measures. 

The  steps  above  form  an  analysis  framework  that  is  proposed  and  being  tested.  It 
is  developed  so  that  it  can  incorporate  JANUS  (or  perhaps  JCATS  or  others)  force-on- 
force  simulations  (Step  4).  JANUS  is  a  high-resolution  force  on  force  simulation 
program  that  models  individual  entities  in  combat  situations.  The  output  of  these 
simulations  is  used  in  analysis  of  performance  under  specific  conditions  and  situations; 
two  important  features  that  JANUS  takes  into  account  in  simulations  are  attrition  and 
terrain.  The  screen  capture  in  the  figure  below  shows  the  high-resolution  nature  of 


JANUS,  i.e.,  individual  entities  are  represented  for  opposing  forces  and  terrain  is 
factored. 


Qualnet  is  a  communication  network  modeling  tool  used  to  run  the  experiments 
documented  in  this  report.  It  has  origins  with  the  DARPA  GloMo2  project.  Qualnet 
developers  claim  it  to  be  an  accurate  and  efficient  communication  software  used 
commercially  and  by  some  within  DoD  to  test  the  bounds  of  current  communication 
systems  as  well  as  to  experiment  with  new  communication  system  ideas.  Simulation  run¬ 
times  vary  based  on  the  network  size,  traffic,  and  other  factors.  For  the  experiments 
reported  in  this  chapter,  simulation-time  to  real-time  ratios  were  consistently  less  than  5:1 
(real-time  to  simulation-time).  Like  Janus  (and  JCATS),  Qualnet  takes  into  account 
terrain  using  map  data  (e.g.,  DTED1). 

2see  B.  Leiner,  R.  Ruth,  S.  Ambatipudi,  "Goals  and  Challenges  of  the  DARPA  GloMo  Program," 
IEEE  Personal  Communications,  Dec.  1996. 


Figure  2:  Qualnet  screen  capture 


The  framework  proposed  in  this  section  is  one  that  will  utilize  both  these  two 
high-resolution  simulation  tools  in  order  to  provide  analysis  of  combat  effectiveness  that 
dynamically  factors  communication  performance.  This  merging  could  be  accomplished 
by  directly  inserting  Qualnet  into  JANUS,  but  run-time  is  an  important  issue  for  high- 
resolution  simulations.  Instead,  meta-models  are  developed  to  capture  the  effects  of  the 
technology  and  other  factors  of  interest,  and  can  realistically  represent  network 
performance  in  combat  without  adding  the  overhead  time  of  directly  using  a 
communication  simulator. 


Figure  3:  High-resolution  Modeling  is  Incorporated 


Why  A  Meta-modeling  approach? 


A  communication  network  simulator  is  a  mechanism  that  turns  factors  (e.g., 
frequency,  number  of  UAVs,  radio  power,  node  density,  frequentness  of  data 
transmissions,  etc.)  into  performance  responses  (e.g.  packet  delivery  ratio,  end-to-end 
delay).  In  this  sense,  the  network  simulator  is  a  function.  A  large  number  of  simulation 
runs  can  be  used  to  provide  enough  information  so  that  an  algebraic  expression  of  the 
performance  response  is  formulated  as  a  function  of  the  factors  of  interest  via  regression 
analysis.  This  is  called  meta-modeling  (Law  and  Kelton,  1992). 

Communication  network  simulation  is  complex  and  time  consuming.  A 
communication  network  simulator  could  be  integrated  directly  into  a  force-on-force 
simulator  so  that  operational  performance  and  communication  performance  are  both 
determined  with  a  high  degree  of  resolution.  But,  there  is  an  added  cost  in  terms  of 
computational  time  and  complexity  of  such  a  pairing.  Meta-modeling  is  an  alternative.  A 
meta-model  can  capture  the  effects  of  the  technology  and  other  components  of  interest 
and  can  realistically  represent  network  performance.  This  research  effort  proposes  that  it 
can  be  incorporated  into  combat  simulations  so  that  communication  (and  information 
dissemination)  performance  can  be  dynamically  represented. 

Factors  of  Impact  Addressed 

There  are  a  large  number  of  relevant  factors  that  impact  communications 
performance  and  C4ISR  performance  in  general.  A  small  subset  of  key  factors  was 
selected  to  test  the  proposed  framework  and  modeling  approach.  They  are  as  follows. 

1.  Terrain  (elevation  variation,  foliage,  etc.) 

2.  User  (radio)  throughput  capability  (in  Kbps) 

3.  Signal  frequency 


4.  Message  data  rate 

5.  Presence  of  UAV’s  to  serve  as  reconnects 

6.  Density  of  nodes  on  the  same  network 

7.  Distance  between  nodes 

8.  Line-of-sight  between  nodes 


Terrain  Considered 


t  ThJ  D -I 


Figure  4:  Terrain  Boxes  (areas) 

The  roughness  (or  varying  elevation3)  impacts  line-of-sight,  signal  attenuation,  and 
other  factors  that  can  severely  enhance  or  degrade  communication  capability.  Four 
terrain  boxes  were  used  in  this  experimentation  process.  They  are  shown  in  the  figure. 
Area  4  is  shown  as  the  lower  right  box.  It  has  a  40km  x  40km  land  area  that  is  not 


3  See  Brennan  (1987)  for  a  discussion  of  use  of  terrain  elevation  data  in  metamodels  of 
communication  performance. 


contiguous  with  the  other  boxes  and  is  ~60  km  away  from  the  box  labeled  Area  2  in  the 
figure. 

The  approach  is  to  model  network  performance  inside  each  of  the  individual  boxes 
via  simulation.  The  boxes  vary  in  size  and  terrain  roughness  as  shown  below  in  the 


figures  that  follow. 
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Elevation  Data  for 
Terrain  Box  for  UA  #2 
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Figure  5:  Elevation  Details  of  Terrain  Boxes 
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Signal  Frequency 

Lower  transmission  frequencies  (e.g.,  200-400  Mhz)  propagate  across  rough 
terrain  and  through  foliage  better  than  higher  frequencies  (e.g.,  2-38  GHz).  However, 
there  is  more  availability  at  higher  frequencies  in  terms  of  spectral  allocation. 
Furthermore,  higher  frequencies  are  less  detectable.  Candidate  bands  being  considered 
for  JTRS  (Boeing,  2002)  include  the  following: 


1)  225-400  MHz 


User  Transmission  Rate 


2)  1350-1390  MHz 

3)  1755-1850  MHz 

4)  2200-2290  MHz 

5)  2400-2500  MHz 

The  rate  that  a  radio  transmits  data  varies.  An  existing  radio  like  the  NTDR  has  a 
reported  user  data  rate  of  288  Kbps  (maximum).  According  to  the  JTRS  Wideband 
Network  (WNW)  Functional  Description  Document  (JTRS  Joint  Program  Office,  2001), 
“the  JTRS  WNW  shall  support  user  throughputs  greater  than  2  Mbps  as  a  Threshold  and 
5  Mbps  as  an  Objective”.  In  our  experimentation,  we  consider  2  Mbps  and  6  Mbps  user 
data  rates  for  the  radios  modeled. 

Message  Data  Rate 

Data  rates  vary  by  message  type.  Voice  data  can  be  as  low  as  8Kbps.  Video 
teleconference  can  be  256  Kbps  or  higher.  Streams  of  live  video  can  be  at  1  Mbps  and 
higher.  The  size  of  a  COP  and  the  frequentness  of  its  dissemination  could  span  all  of  the 
aforementioned  rates.  Thus,  simulation  tests  will  also  have  to  span  these  data  rates. 

Presence  ofUAVs  as  Reconnects 

Vertical  nodes  that  support  communication  relay  can  be  in  the  form  of  fixed  wing 
aircraft,  unmanned  aerial  vehicles  of  various  sizes  and  operating  altitudes,  and  high 
altitude  airships  (>  60,000  ft).  For  the  terrain  boxes  identified  in  this  chapter,  we  consider 
0,  4,  and  8  UAVs  as  dedicated  relay  platforms.  One  objective  of  this  research  effort  is  to 
determine  the  required  number  of  relay  platforms  needed  to  provide  the  required  reliable, 
accurate  communications  connectivity  that  enhances  warfighter. 


? 


Satellite 
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Figure  6:  Concept  for  a  Multi-layer  Communication  Architecture-  Signal  Center  (Kioutas, 

2003) 

Density  of  Nodes  on  the  Same  Network 

The  terrain  boxes  considered  cover  areas  of  25km  x  25km  in  two  cases  and  40  km 
x  40km  in  the  two  others.  Nodes  are  dispersed  so  that  there  is  experimentation  with  36 
nodes,  72  nodes,  and  145  nodes.  This  means  that  densities  from  0.0225  nodes/km2  to  .232 
nodes/km2  are  considered.  A  model  with  density  as  a  factor  facilitates  a  model  response 
that  is  dynamic,  i.e.,  changes  with  attrition.  This  is  a  key  attribute  of  our  approach  and 
highly  appropriate  given  its  intended  use  as  part  of  a  force-on-force  combat  simulation 
exercise. 

Distance  Between  Nodes 

The  nodes  in  the  terrain  boxes  are  randomly  dispersed  such  that  the  range 
between  individual  nodes  range  from  very  close  (<  0.5  km)  to  distant  (35  km). 


Line  of  Sight 


The  line  of  sight  between  two  nodes  represents  the  condition  of  whether  or  not 
there  is  any  obstruction  between  the  path  of  a  sender  and  a  receiver  of  a  message.  It  is 
impacted  by  the  relative  elevation  of  the  node-pairs  and  of  course  the  surrounding  terrain. 


Figure  7:  Assessing  line-of-sight  (LOS) 

In  the  figure,  the  height  of  the  destination  relative  to  the  origin  is  shown. 
Intermediate  points  (dl,  d2,  d3,  etc.)  along  the  path  between  origin  and  destination,  and 
the  corresponding  heights,  determines  whether  a  pair  enjoys  line-of-sight.  The  example 
in  the  figure  depicts  a  non-line-of-sight  condition. 

Line-of-sight,  based  on  relative  elevation  data,  between  a  single  node-pair,  may  not 
be  suitable  alone  as  an  indicator  of  communication  performance.4  This  is  because  a 
network  may  facilitate  a  simple  path  that  involves  one  or  more  hops  through  intermediate 
nodes.  Perhaps  a  more  useful  factor  measures  the  average  line-of-sight  a  given  node  has 
with  all  of  its  neighbors.  The  figure  below  provides  an  example  calculation  of  such  a 


measure  (In  the  figure,  a  blocked  line  of  sight  equates  to  1  and  a  completely  clear  line  of 


sight  is  zero.) 


Figure  8:  Simple  Measure  of  Average  Line-of-Sight 

Later  in  this  report,  a  description  is  given  of  measurements  that  were  taken  of  the 
average  line-of-sight  a  node  has  with  all  of  its  neighbors.  This  factor  appeared  to  improve 
the  fit  enough  to  suggest  that  it  is  a  useful  measure.  More  testing  and  examination  is 
needed. 

3.  General  Experiments 

Simple  Simulation  Experiments  to  Address  Scalability 
A  number  of  simulation  experiments  were  performed  to  generally  assess 
communication  network  capabilities.  We  described  three  sets  of  simulation  experiments. 
In  the  first,  Qualnet  is  used  in  a  terrain-less  set-up.  Nodes  are  distributed  at  various 
spacing.  The  network  is  fully-connected,  i.e.,  each  node  is  connected  to  another  as  shown 
below. 


4  A  1987  CECOM  report  (Brennan,  1987)  reached  a  similar  conclusion. 


Figure  9:  Depiction  of  a  (Nearly)  Fully-connected  Network 


The  results  from  these  tests  are  shown  below.  They  highlight  one  of  the  most 
critical  aspects  of  a  C4ISR  (or  battle  command)  network  -  scalability.  In  the  figure 
below,  a  data  rate  is  offered  (transmitted)  by  each  node.  For  the  small,  three-node 
network  shown  as  the  top  line,  the  load  can  be  handled,  i.e.,  what  goes  in  is  what  goes 
out.  For  larger  networks  like  the  bottom  curve,  which  represents  a  49-node  network, 
throughput  above  even  a  low  rate  (e.g.,  24Kbps),  is  limited. 

Fully  Connected  Network:  Poor  Scalability 


Data  Rate  in  Kilobits  Per  Second  (Offered) 


Figure  10:  Assessing  Scalability 


This  is  a  known  result  in  networking  (See  Gupta  and  Kumar,  1999).  It  suggests 
that  fully  connected  networks  like  the  one  shown  in  the  figure  will  only  exist  in  small 
numbers  and/or  with  minimal  data  traffic.  Hierarchical  or  regionalized  information 
dissemination  will  almost  certainly  be  required.  This  is  depicted  in  the  figure  below. 


Figure  11:  Regionalized/Hierarchical  Network 

The  point  made  by  the  results  of  these  simple  experiments  is  to  demonstrate  the 
inherent  scalability  issue  associated  with  wireless  communication  in  general. 
Aggregation  or  clustering,  in  terms  of  the  communication  architecture,  has  always  been 
one  approach  to  addressing  this  concern.  A  fully  decentralized,  fully-connected 
architecture  in  which  each  node  communicates  with  every  other  node  is  only  practical  for 
small  networks  and/or  networks  that  can  exist  with  very  low  data  rates  (e.g.,  24Kbps).  A 
regionalized  approach  to  information  dissemination  (and/or  data  fusion)  is  more  practical 
(and  probably  a  necessity)  for  larger  networks. 

A  second  set  of  simulation  experiments  was  exercised.  This  time,  using  terrain 
and  force  structure  that  reflects  a  (future)  company  sized  Army  unit.  The  unit  was 
simulated  with  varying  message  traffic.  Specifically,  the  time  a  UAV  sensor  is  used,  as  a 
percentage  of  the  time  the  sensor  is  transmitting  data,  was  varied  as  well  as  the 


frequentness  of  the  COP  update  message.  A  specific  terrain  map  was  used.  It  is  shown 
below. 


A  small  company  sized  force  was  emplaced  in  this  specific  region  as  shown  in  the 
figure  below. 
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Figure  13:  Generic  Force  Laydown 
The  simulation  experiments  consisted  of  the  following  steps: 

1.  Vary  UAV  Sensor  data  usage 

a.  0%  Tx  Active 

b.  25%  Tx  Active 


c. 


75%  Tx  Active 


d.  100%  Tx  Active 


2.  Vary  COP  Update  Freq. 

a.  Very  Frequent  (10  times  /  min) 

b.  Infrequent  (once  every  2  minutes) 

3.  Assess  Delay  &  Completion  Rate 

a.  For  COP  Message 

b.  For  UAV  Sensor  data 

The  data  distribution  included  COP  messages  going  out  and  SA  message  coming 
in  as  shown  in  the  figure  below. 


Figure  14:  UAV  Sensors  Transmitted  Imagery  Back  to  a  Command  Vehicle 


Figure  15:  A  Command  Vehicle  Transmitted  a  COP  Update  Out 

Raw  data  from  the  simulation  runs  is  shown  in  the  figure  below.  The  figure  suggests  that 
when  sensor  usage  is  below  50%  of  the  time  and  COP  updates  are  held  to  no  more  than 
once  a  minute,  then  this  allowed  completion  rates  to  be  above  75%.  There  appears  to  be  a 
trade-off  with  the  degree  of  sensor  usage. 
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Figure  16:  Simulation  Results 
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Figure  17:  Tradeoffs 


The  figure  suggests  that  sensor  usage  below  50%  of  the  time  and  COP  updates  held 
to  no  more  than  once  a  minute  was  sufficient  in  allowing  completion  rates  to  be  above 
75%.  There  appears  to  be  a  trade-off  with  the  degree  of  sensor  usage. 

These  results  and  observations  were  somewhat  intuitive.  The  need  to  transmit 
sensor  data  and  conflicted  with  the  need  to  transmit  cop  update  data,  when  sharing  a 
communication  channel.  The  following  observations  cited  are  drawn  from  simulation 
results:  (1)  Less  frequent  COP  updates  and  limited  sensor  usage  are  needed  to  get 
adequate  network  performance;  neither  will  be  able  to  unconstrained  (2)  aggregated 
architectures  for  information  dissemination  and  data  fusion  will  be  required  to  handle  the 
scalability  issues.  One  resolution  to  the  1st  observation  is  to  put  sensor  links  and  COP 
update  (SA)  links  on  separate,  non-contending  channels  (assuming  spectrum  availability 
is  not  an  issue).  Putting  sensor  data  on  separate  data  links  (channels)  seems  necessary. 


4.  Experiments  To  Synthesize  Meta-models 


A  factorial  design  (Law  and  Kelton,  2000)  was  employed  for  the  factors  described 


earlier  (transmission  frequency,  number  of  UAYs,  etc.)  and  shown  in  the  figure  below. 
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Figure  17:  Factors  and  Responses 

The  number  of  nodes  was  set  such  that  these  experiments  could  represent  a 
Battalion  sized  force  at  various  levels  of  attrition.  An  example  is  shown  in  the  table 
below.  The  table  shown  is  not  the  actual  experiment  designed  but  is  shown  to  indicate 
how  the  factors  were  varied;  the  last  two  columns  represent  the  responses  sought,  which 
are  PDR  (packet  delivery  ratio)  and  end-to-end  delay.  Thousands  of  simulations  were  run 
for  each  area  of  interest. 


Table  1:  Example  Design  of  Experiments 


# 

frequency 

# 

UAVs 

density 

Radio 

Power 

Radio 

Throughput 

Data 

Rate 

PDR 

Delay 

1 

2  GHz 

0 

145 

2  W 

2Mbps 

16Kbps 

result 

result 

2 

1  GHz 

4 

72 

20  W 

2Mbps 

32Kbps 

result 

result 

3 

0.4  GHz 

8 

36 

1600  W 

2Mbps 

64Kbps 

result 

result 

4 

0.2  GHz 

0 

145 

2  W 

6  Mbps 

160Kbps 

result 

result 

5 

2.5  GHz 

4 

72 

20  W 

6  Mbps 

320Kbps 

result 

result 

6 

0.1  GHz 

8 

36 

1600  W 

6  Mbps 

533Kbps 

result 

result 

Figure  18:  Node  Placement  on  Terrain  Box  1 

Experiments  were  conducted  with  a  small  Bn-sized  unit.  Approximately  145 
nodes  randomly  scattered  across  a  25km  by  25km  area  in  the  Macedonia-Serbia  area  as 
shown  in  the  figure  above  that  is  referred  to  as  terrain  box  1 .  The  COP  update  (in  terms 
of  size  and  frequency)  was  varied.  Network  performance  was  observed  as  function  of 
frequency,  mobility,  UAV  usage,  etc.  The  terrain  is  fairly  flat. 

All  of  the  experiments  were  20-minute  Qualnet  simulations.  These  experiments 
focused  on  COP  update  messages.  The  COP  update  is  dissemination  of  information  by 
one  informed  node  (e.g.,  the  commander  control  vehicle)  to  everyone  else  giving  them 
information  about  the  whereabouts  of  all  other  relevant  vehicles  (as  well  as  other 


information  including  perhaps  enemy  forces  and  weather  and  terrain  data).  The 


dissemination  “Architecture”  is  as  shown  in  the  figure  below.  The  size  of  a  COP  update 
is  varied  as  it  can  be  made  small  or  large  and  sent  frequently  or  infrequently. 


Figure  19:  Information  Dissemination 

There  are  no  other  messages  being  transmitted  on  the  network  for  these 
experiments.  This  is  equivalent  to  the  COP  update  messages  receiving  its  own  dedicated 
channel  on  the  system.  For  each  experiment,  the  variables  of  interest  were  set  and 
recorded  and  then  the  simulation  was  run;  network  performance  measures  were  then 
recorded. 

Performance  Measures  (Responses) 

The  performance  measures  being  tracked  are  packet  delivery  ratio5  (PDR)  and 
delay6.  PDR  is  defined  as  the  number  of  packets  received  by  a  node  divided  by  the 
number  of  packets  sent  to  it;  the  packet  delay  is  defined  by  the  time,  in  seconds,  the 
packet  was  received  minus  the  time  that  the  packet  was  sent.  One  or  more  (n)  packets 
make  up  a  message.  So,  the  likelihood  that  a  message  gets  through  is  related  to  the 
likelihood  that  n  packets  get  through. 


5  PDR  =  #  of  packets  received  /  #  of  packets  sent 

6  Delay  =  time  packet  received  -  time  packet  sent 


Likelihood  Message  Received  For  Various  Packet  Delivery  Ratios  (p) 


Figure  20:  Packet  Delivery  Ratio  and  MCR 

Clearly,  multi-packet  messages  require  high  packet  delivery  ratios  in  order  to 
ensure  a  high  message  completion  rate.  The  chart  in  the  figure  assumes  that  message 
completion  rate  has  the  probability  distribution  of  the  negative  binomial  distribution.7 


Factors 

The  factors  of  interest  were  chosen  previous  to  running  the  experiments  as  well  as 
the  possible  levels  at  which  they  will  impact  the  network.  These  factors  are  varied 
systematically  throughout  all  of  the  experiments  as  alluded  to  in  the  table.  The  specific 
ranges  for  the  frequency  of  the  transmissions  were  varied  across  6  levels,  between  0.1 


7  The  negative  binomial  distribution  is  the  probability  distribution  of  the  number  of  trial  needed  to 
get  a  fixed  number  of  success.  It  has  two  parameters:  the  number  of  successes  and  the  success  probability. 
So,  if  the  probability  of  getting  a  single  packet  transmitted  is  p  (or  pdr)  then  the  negative  binomial 
distribution  relates  this  to  n-packet  messages  as  n  trials.  Specifically,  the  probability  that  rth  success  (rth 
packet  gets  transmitted  successfully)  occurs  on  the  xth  try=C(x-l,r-l)pAr(l-p)A(x-r),  where  C  is  the  binomial 
coefficient 


and  2.5;  the  Cop  message  data  rate  having  7  levels  between  16.0  kilobytes  per  second  and 
533  kilobits  per  second,  the  presence  of  unmanned  aerial  vehicles  (UAVs)  to  serve  as 
reconnects  varied  between  0  and  8,  the  network  density  varied  between  145  and  36  nodes 
in  the  25km  by  25km  area,  and  the  distance  between  the  receiver  and  the  sender  varied 
based  on  the  node  pair. 

In  these  experiments  the  COP  update  message  was  sent  with  a  multicasting 
transmission  process,  this  means  that  every  node  is  told  the  message  and  then  told  to  tell 
its  neighbors.  This  means  that  a  particular  node  can  receive  the  message  from  the 
commander  (or  other  central  data  source)  directly  or  indirectly  from  an  intermediate 
node.  UAVs  that  are  present8  to  serve  as  reconnects  act  as  extra  nodes  that  do  not  receive 
messages  themselves  but  can  act  as  relays  for  messages  to  hop  on. 

Network  density  relates  to  potential  attrition.  The  experiments  included  runs  with 
145  nodes.  Other  experiments  were  run  at  -50%  of  that  number  (72).  Other  experiments 
were  run  with  36  nodes.  This  was  done  in  order  to  capture  the  impact  of  the  density  of 
nodes  in  the  same  box  on  the  same  communication  networks. 

The  as-the-crow-flies  distance  between  nodes  is  measured  from  a  node  centered 
in  the  region  and  are  fixed  at  the  beginning.  The  distance  from  the  vehicle  transmitting 
messages  to  the  other  nodes  receiving  the  messages  ranges  from  -  1km  to  18  kilometers 
away  in  areas  one  and  two.  The  range  expands  to  29  km  in  areas  three  and  four. 

It  is  noted  that  terrain  has  a  great  impact  on  communication  systems  and  thus  the 
models  are  specific  to  the  terrain  box.  The  experimental  results  apply  to  terrain  box  1 


(area  1). 


Products:  Predictive  Models 


A  predictive  model  of  the  packet  delivery  ratio  can  be  developed  from  data  from 
simulation  runs  as  shown  in  the  figure  below.  The  logit  function  is  convenient  because  it 


P 

produces  a  value  between  0  and  1.  It  is  defined  as  logit(p)  :=  log  ( - ).  Once  a  model 

1  ~P 


for  the  logit(PDR)  is  determined,  the  actual  PDR  (labeled  p)  is  calculated  as  follows: 


exp(logft(p)) 
(exp(logif(p))  +  l 


Logit  (PDR)  =  P0  + 


|31(Frequency)+p2(UAVs)+. . 


Other 

First-Order 

Terms 


P3(FrequencyxUAVs)+. . . 


Other 
Second-Order 
Terms 


Other  Higher  Order  Interactions 


Figure  21:  Generic  Form  of  Metamodel 


Before  proceeding,  it  is  important  to  assess  whether  or  not  simulation  experiment 
data  is  amenable  to  being  fit  to  such  a  closed  form  expression.  A  simpler  model  of 
averaged8 9  performance  was  fit  via  regression  for  area  2  (25km  by  25km)  and  area  3  (40 
km  by  40km).  The  plots,  shown  below,  suggest  a  good  fit.  Similar  results  were  observed 


8  The  UAVs  present  in  the  experiments  are  placed  in  a  diamond  formation  in  the  box. 

9  By  averaged,  it  is  meant  that  distances  between  communication  nodes  specific  node  pairs  is  not  a 
parameter,  i.e.,  distance  between  node-pairs  is  not  considered  as  a  factor 


for  models  of  end-to-end  delay.  The  fits  were  only  slightly  improved  if  five-parameter 
interactions  were  captured,  which  resulted  in  r-square  value  of  approx.  0.91. 
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Figure  22:  Area  2  ’’Averaged”  Model  for  Packet  Delivery  Ratio  Two  Way  Interactions  r-square 

values  of  0.896 
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Figure  23:  Area  3  ’’Averaged”  Model  for  Packet  Delivery  Ratio  (R-square  is  0.86)  Capturing  up  to 

Two-parameter  Interactions 

Appendix  A  provides  models  for  all  four  areas.  These  models  were  not  created 


with  the  averaged  data  but  rather  with  node-pair- specific  data.  Thus,  distance  between  a 


particular  node  pair  is  a  factor  in  the  models.  The  r-square  values  are  shown  in  the  table 
below. 


Table  2:  Evaluating  The  Fits  for  2-way  Parameter  Interaction 


Area/ 

Equation 

Adj.  R-Square 
Value  for  PDR 

Adj.  R-Square 
Value  for  Delay 

1 

0.715 

0.755 

2 

0.749 

0.817 

3 

0.725 

0.813 

4 

0.706 

0.800 

Note:  The  fits  could  have  been  slightly  improved  by  representing  the  four-way 
parameter  interaction  in  the  model.  An  even  better  fit  was  achieved  by  adding  an 
additional  factor  called  LOS  that  represents  the  degree  of  “connectivity”  associated  with 
individual  nodes  based  on  its  line-of-sight  with  other  nodes.  LOS  was  examined  for 
Areas  2  and  3. 


Table  3:  Evaluating  the  Fits  for  4-way  Parameter  Interaction 


Area 

Equation 

Adj.  R-Square 
Value  for  PDR 

Adj.  R-Square 
Value  for  Delay 

Adj.  R-Square 
Value  for 

PDR,  LOS 

Adj.  R-Square 
Value  for 

Delay,  LOS 

1 

0.72 

0.78 

- 

- 

2 

0.75 

0.84 

0.77 

0.85 

3 

0.73 

0.84 

0.78 

0.86 

4 

0.71 

0.82 

- 

- 

Interpreting  The  Results 

Analysis  using  the  models  synthesized  (see  Appendix  A  for  details)  is  provided  in 
the  following  subsections.  One  of  the  many  observations  facilitated  by  the  synthesized 
models  is  the  critical  role  of  dedicated  UAVs  for  communication  relay  and  connectivity. 
Contour  plots  are  shown  below  that  use  data  from  simulations  of  area  3.  These 


contour  plots  are  taken  from  the  data  generated  through  thousands  of  simulation 


experiments.  On  one  axis  is  the  distance  between  a  potential  sender  and  receiver.  On  the 
other  access  is  a  measure  of  the  connectivity  of  line-of-sight  between  two  such  nodes.  A 
high  value  (e.g.,  1)  suggests  that  the  line-of-sight  is  blocked.  In  the  figures,  the  area  in  the 
lower  left  corner  represents  communication  that  is  short  in  distance  and  enjoys  a  clear 
line  of  sight.  Intuitively,  performance  here  should  be  better  than  in  the  upper  right  hand 
area  where  distances  are  longer  and  line-of-sight  is  more  blocked. 

As  shown  in  Figure  25,  with  no  UAVs,  performance  is  poor  for  many  node  pairs, 
especially  those  that  don’t  enjoy  the  short  distance  and  relatively  good  line  of  sight  with 
neighboring  nodes.  In  Figure  26,  many  of  the  node  pairs  that  were  situated  poorly  (e.g., 
ones  that  are  distant  and  have  poor  line  of  sight  with  neighboring  nodes)  now  operate 
with  much  improved  performance. 
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Figure  24:  Performance  with  0  UAVs  (packet  delivery  ratio)  given  distance  and  line-of-sight  measure 
for  Area  3 
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Figure  25:  Performance  with  8  UAVs  (packet  delivery  ratio)  given  distance  and  line-of-sight  measure 
for  Area  3 


Other  selected  observations  are  described  as  follows. 


High  Radio  User  Throughputs  Help 
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Figure  26:  Area  1  Results  with  High  User  Throughput  Radios 

Communication  performance  shown  in  Figure  27  is  strong.  This  is  because  the 
radio  modeled  are  using  a  high  user  throughput  (6  Mbps),  high  power  radio  (62  dbm).  As 
a  baseline,  the  JTRS  radio  cluster  1  user  throughput  will  be  2  Mbps  (maximum).  The 
remainder  of  this  section  will  use  synthesized  models  that  assumed  a  network  of  2Mbps 
radios  at  a  transmit  power  of  43dBM. 

In  contrast,  simulations  results  (below)  with  2  Mbps  radios  are  not  as  good  across 
all  data  rates  as  the  higher  user  throughput  radios.  Figure  28  shows  results  using  radios 
with  radio  power  of  20  W  (43  dbm).  The  performance  degrades  with  data  rate  more 
significantly. 

Key  Advantages  at  Certain  Frequencies 

The  raw  data  shows  that  frequency  agility,  i.e.,  the  ability  to  selectively  transmit 


at  various  frequencies,  is  important. 
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Figure  27:  Impact  of  UAVs  in  Area  2  at  f=0.4  GHz 
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Figure  28:  Frequency  Has  An  Impact  on  Performance 

Figure  29  shows  that  performance  at  2.5  GHz  is  significantly  lower  than  at  .4  GHz 
(Figure  5.30).  The  reason  for  this  is  explained  concisely  in  Vanderau  et  al  (1998)  as 
follows:  The  way  a  radio  signal  propagates  through  the  airwaves  is  different  at  higher 


frequencies  (F)  than  at  lower  ones.  Signal  loss  between  a  radio  transmitter  and  receiver 
increases  with  frequency,  e.g.  the  loss  is  proportional  to  1/F2.  Quoting:  “While  all  radio 
waves  tend  to  move  in  straight  lines,  higher  frequencies  are  blocked  more  sharply  by 
terrain  or  buildings”.  Generally  speaking,  as  frequency  increases,  radio  coverage 
decreases.  This  is  not  always  detrimental  as  a  decreased  range  can  translate  into 
decreased  interference  in  a  dense  network.  In  fact  higher  frequencies  are  more  difficult  to 
detect.  However,  UAVS  did  improve  capabilities  at  this  frequency.  Because  channels  at 
2.5GHz  and  higher  are  likely  more  available  in  terms  of  spectrum  constraints, 
performance  issues  at  these  and  higher  frequencies  cannot  be  ignored. 

To  get  an  idea  of  the  performance  impact  of  various  factors,  the  table  below 
varies  key  factors  and  reports  on  the  maximum  data  rate  achievable  while  maintaining  a 
90%  packet  delivery  ratio.  Note:  table  used  models  of  average  performance. 


Table  4:  Results  From  a  Sample  of  Experiments 


Area 

#  of  UAVs 

Density 

Nodes/km2 

Frequency 

90%  PDR 

Data  Rate 

2  (25  x25) 

8 

.12 

2.5  GHz 

270  Kbps 

2  (25  x25) 

4 

.12 

2.5  GHz 

255  Kbps 

2  (25  x25) 

0 

.12 

2.5  GHz 

240  Kbps 

2  (25  x25) 

8 

.12 

0.4  GHz 

320  Kbps 

2  (25  x25) 

4 

.12 

0.4  GHz 

345  Kbps 

2  (25  x25) 

0 

.12 

0.4  GHz 

375  Kbps 

2  (25  x25) 

8 

.06 

2.5  GHz 

110  Kbps 

2  (25  x25) 

4 

.06 

2.5  GHz 

40  Kbps 

2  (25  x25) 

0 

.06 

2.5  GHz 

- 

Note:  Data  rate  capability  impacts  the  size  of  the  COP  message  and  the 


frequentness  of  the  COP’s  dissemination 


UA  Vs  Are  Critical  for  Less  Dense  Networks 
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Figure  29:  Impact  of  UAVs  is  Pronounced  Especially  for  Sparse  Forces 
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Figure  30:  Impact  of  UAVs 


5.  What  the  Models  Tell  Us 


The  meta-models  presented  follow  intuition  in  some  respects: 

(1)  Data  rate  drives  the  performance.  Depending  on  other  parameters, 
network  performance  will  degrade  precipitously  at  a  certain  level  of 
offered  traffic. 

(2)  Higher  network  density  and  the  presence  of  UAVs  increase  the 
likelihood  of  a  message  being  completed. 

(3)  While  high  data  rates  start  to  tax  a  network  at  some  level,  larger 
distances  between  nodes  and  high  signal  frequencies  decrease 
likelihood  of  messages  getting  through  and  increase  end-to-end  delay. 

Other  observations: 

(4)  The  presence  of  UAVs  help  network  performance,  especially  for  the 
transmission  at  higher  signal  frequencies.  However,  there  is  a  limit  to 
the  number  of  additional  UAVs  that  enhance  performance;  too  many 
can  hurt  network  performance.  UAVS  help  when  they  are  needed  but 
could  be  a  nuisance; 

(5)  Other  factors  (e.g.  network  density)  and  technologies  (e.g.,  frequency 
agile  radios,  SATCOM,  etc.)  may  obviate  the  need  for  large  numbers  of 
UAVs  at  the  tactical  level. 

6.  Observations  and  Conclusions 

The  U.S.  Army  is  developing  a  fighting  force  intended  to  be  deployable,  rapid¬ 
reacting,  lethal,  and,  foremost,  able  to  maintain  situation  awareness  and  responsiveness  to 
a  degree  that  it  can  shape  the  battlefield  and  chose  the  battles  where  it  may  apply 
overwhelming  force.  Superior  C4ISR  will  be  required.  This  study  has  developed  a 


framework  for  analyzing  the  marginal  impact  of  communication  technology  and 
architectural  options  for  the  future  force. 

At  the  tactical  level  (brigade  and  below),  the  performance  of  the  C4ISR  or  battle 
command  network  is  highly  sensitive  to  technology  detail  and  scenario  specifics.  Terrain 
plays  a  large  role.  Antenna  size  and  type,  radio  frequency,  and  mobility  all  have 
significant  impacts  on  technical  performance  in  terms  of  connectivity,  delays  and 
message  completion  rates.  Communication  network  simulation  runs  were  used  to 
synthesize  meta-models  to  capture  communication  performance.  Analysis  using  the 
models  thus  far  has  yielded  some  preliminary  observations  that  are  summarized  as 
follows:  (1)  High  capacity  radios  for  future  forces  will  be  very  important  towards  good 
performance  (>  5  Mbps  user  throughput);  however,  spectral  availability  issues  are  key 
towards  higher  data  rate  radios  (see  Joe  and  Porche  (2004))  and  will  be  a  limiting  factor. 
(2)  This  suggests  the  importance  and  advantage  of  future  concepts  like  frequency  agile, 
cognitive  radios.  (3)  Near  future  radios  (limited  to  1-2  Mbps  user  throughputs)  on 
vehicles  will  require  significant  UAV  presence  to  ensure  reliable  situational  awareness 
network  performance.  This  is  especially  relevant  if  smaller  forces  are  expected  to  be 
responsible  for  larger  and  larger  areas  of  operation. 

7.  Discussion:  On-going  and  Future  Work 

These  analyses  have  provided  insight  into  battlefield  network  communications 
performance  parameters.  Only  a  handful  of  factors  of  interest  were  considered  but  the 
analysis  provided  a  possible  template  for  modeling  data  of  this  type  in  future  simulation 
efforts.  The  scope  will  expand  to  more  variables  of  impact  to  explore  and  more 
complicated  scenarios.  Hopefully  the  models  described  above  will  easily  expand  as  these 


are  added  to  the  variable  list,  but  continued  diagnostics  are  needed  to  ensure  that  the 
proper  model  is  being  fit  to  the  data. 

When  these  models  are  inserted  into  force-on-force  simulation  exercises, 
questions  related  to  the  impact  of  communication  on  operations  will  be  addressable.  For 
example:  (1)  What  is  the  marginal  impact  of  UAV  assets  on  warfighter  effectiveness? 
What  is  the  robustness  of  the  situational  awareness  network  for  a  given  force  and  a  given 
area  of  operation? 


Figure  31:  Next  Step:  QUALNET  Derived  Model  Inserted  in  Combat  Simulator 

This  figure  is  a  more  detailed  overview  of  the  project’s  goals  for  interfacing  a 
force-on-force  simulator  (e.g.,  Janus)  with  the  communication  meta-model  developed 
through  Qualnet.  A  set  of  experiments  will  be  conducted  with  the  force-on-force  combat 
simulator  using  all  four  terrain  boxes  with  various  levels  of  message  transmission 
activity  as  well  as  different  nodal  density  levels  and  different  numbers  of  UAVs. 
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Appendix 

Appendix  A.  Communication  models  for  Terrains 

The  closed  form  expressions  described  in  this  appendix  represent  models  of 
communication  for  areas  one  through  four.  The  factors  of  interest  in  the  models  are 

•  Frequency  (f) 

•  Radio  power  (p) 

•  Density  (d) 

•  Data  rate  (rt) 

•  Number  of  UAYs  (u) 

•  Distance  between  node  pairs  (ds) 


The  responses  modeled  are 

•  Packet  delivery  ratio  (PDR) 

•  End  to  end  delay  (delay) 

The  four  models  described  were  fit  via  regression  analysis  using  data  from 
numerous  simulation  runs  using  Qualnet.  Specifically,  the  data  were  fit  to  the  logit 
function  of  the  packet  delivery  ratio  and  the  log  of  the  end  to  end  delay.  The  adjusted  r- 
squared  values  are  shown  in  Table  A.l. 


Table  A.l:  Evaluating  the  fits  for  2-way  parameter  interaction 


Area/Equation 

R-Square  Value  for  PDR 

R-Square  Value  for  Delay 

1 

0.715 

0.755 

2 

0.749 

0.817 

3 

0.725 

0.813 

4 

0.706 

0.800 

Note:  The  fits  can  be  improved  by  representing  more  than  two-way  parameter  interaction 
in  the  model. 

The  expression  for  a  logit  function  is  shown  below. 

logit(p)  :=  log  (— ^— ) 

1  ~P 


Since  the  regression  analysis  creates  a  model  of  the  logit(pdr),  the  value  of  the  pdr 
is  the  inverse  logit  function.  Thus,  the  inverse  logit  is  of  use  for  this  research  effort.  It  is 
below. 

_  exp(log  it(p)) 

(exp(logzY(p))  +  l 


The  generic  form  of  the  meta-models  being  described  in  this  section  take  the 
following  form: 


Logit  (PDR)  =  p0  + 


Pi  (Distance  )+P2(Powewr)+. . .- 


Other 

First-Order 

Terms 


P3(DistancexUAVs)+. . . 


C  Other 
-s  Second-Order 
Terms 


Other  Higher  Order  Interactions 


The  equations’  formats  can  be  generalized  with  the  coefficients  labeled  ol  through 


07  and  (30  through  (328  as  follows. 


Table  A.2:  Format  of  Data  tables 


Term  Estimate 

Intercept  pO 

ds  pi 

P  P2 

(ds-o1)*(p-o2)  P3 

f  p4 

(ds-o1)*(f-o3)  P5 

(p-o2)*(f-o3)  p6 

d  P7 

(ds-o1)*(d-o4)  p8 

(p-o2)*(d-o4)  P9 

(f-o3)*(d-o4)  P10 

u  pi  1 

(ds-o1)*(u-o6)  pl2 

(p-o2)*(u-o6)  P13 

(f-o3)*(u-o6)  pl4 

(d-o4)*(u-o6)  P15 

rt  pl6 

(ds-o1)*(rt-o5)  P17 

(p-o2)*(rt-o5)  P18 

(f-o3)*(rt-o5)  P19 

(d-o4)*(rt-o5)  p20 

(u-o6)*(rt-o5)  P21 

log(rt)  p22 

(ds-o1)*(log(rt)-o7)  P23 

(p-o2)*(log(rt)-o7)  p24 

(f-o3)*(log(rt)-o7)  P25 

(d-o4)*(log(rt)-o7)  p26 

(u-o6)*(log(rt)-o7)  P27 

(rt-o5)*(log(rt)-o7)  p28 


So  that  the  complete,  generalized  expression  is 

logit(pdr)  =  pO  +  ds*pi  +  p*b2  +  (ds-ol)*(p-o2)*P3  +  f*P4  +  (ds-ol)*(f-o3)*P5  +  (p-o2)*(f-o3)*P6  + 

d*P7  +  (ds-ol)*(d-o4)*P8  +  (p-o2)*(d-o4)*p9  +  (f-o3)*(d-o4)*pi0  +  u*  pll  +  (ds-ol)*(u-o6)*pl2 
+(p-o2)*(u-o6)  *  pl3  +  (f-o3)*(u-o6)*pl4  +  (d-o4)*(u-o6)  *  pl5  +  rt*pl6  +  (ds-ol)*(rt-o5)*pi7  + 
(p-o2)*(rt-o5)*pi8  +  (f-o3)*(rt-o5)*pi9  +(d-o4)*(rt-o5)*  P20  +  (u-o6)*(rt-o5)*P21  +log(rt)*  P22  + 
(ds-ol)*(log(rt)-o7)*  P23  +(p-o2)*(log(rt)-o7)*  P24+(f-o3)*(log(rt)-o7)*P25  +  (d-o4)*(log(rt)-o7)* 
P26  +(u-o6)*(log(rt)-o7)*  P27  +(rt-o5)*(log(rt)-o7)*p28 

where, 

ds  :=  distance  between  node-pairs  in  kilometers 

p  :=  radio  power  in  Decibels 

f  :=  frequency  in  Gigahertz 

d  :=  density  in  nodes  per  square  kilometers 
rt  :=  data  rate  in  kilobits  per  second 

u  :=  number  of  UAVs 


For  example  for  area  1,  the  offset  coefficients  (o*)  are 


ol  = 

9.74145 

o2  = 

45.8749 

o3  = 

1.12383 

o4  = 

0.17633 

o5  = 

199.708 

06  = 

4.09479 

o7  = 

4.71089 

The  equations  presented  above  are  presented  below  in  a  more  readable  form  in  the  tables 
below. 

Table  A.3:  Area  1  (PDR) 


Term 

Estimate 

Intercept 

-7.460954811 

ds 

-0 . 144115681 

P 

0.012725538 

(ds-9. 74145) * (p-45.8749) 

0.000326067 

f 

-0.429489956 

(ds-9. 74145) * (f-1. 12383) 

-0.016828023 

(p-45.8749) *  (f-1. 12383) 

0.002651833 

d 

-2.448472093 

(ds-9. 74145) * (d-0.17633) 

0.352169843 

(p-45.8749) * (d-0.17633) 

-0.009337938 

(f-1. 12383) * (d-0.17633) 

1 . 908550936 

u 

0 . 000494488 

(ds-9. 7 4 145) * (u-4.09479) 

-0.004318525 

(p-45 . 874  9) *  (u-4 .09479) 

5 . 3355E-05 

(f-1. 12383) *  (u-4.09479) 

0.010854036 

(d-0.17633) * (u-4.09479) 

-0.012043543 

rt 

-0.037766834 

(ds-9. 7 4 145) * (rt -199. 708) 

8 . 58345E-05 

(p-45.8  74  9) *  (rt -199. 708) 

-3 . 92614E-05 

( f-1. 12383 ) *  (rt-19  9.7  08) 

0.000651289 

(d-0.17633) * (rt-199.708) 

0.03548329 

(u-4.0  947  9) *  (rt-19  9.7  08) 

2 . 2231 6E-05 

log (rt ) 

3.494608447 

(ds-9. 74145) * ( log (rt) -4.71089) 

-0.008715956 

(p-45.8749) * ( log (rt) -4.71089) 

0.004502034 

(f-1. 12383) * ( log (rt) -4.71089) 

-0.069481796 

(d-0.17633) * ( log (rt) -4.71089) 

-8.424818227 

(u-4.09479) * ( log (rt) -4.71089) 

-0.010022519 

(rt-199.708) * (log (rt) -4 . 71089) 

0.012999863 

Table  A.4:  Area  2 


Term 

Intercept 

ds 

pwr 

(ds-9 . 77858 ) * (pwr-45 . 9117 ) 
f 

(ds-9. 77858) * (f-1. 13673) 
(pwr-45. 9117) *  (f-1. 1367  3) 
d 

(ds-9. 77 858) * (d-0.17612) 
(pwr-45. 9117) * (d-0.17612) 

(f-1 . 13673) * (d-0 . 17612) 
u 

(ds-9. 77858) * (u-4. 03852) 
(pwr-45. 9117) * (u-4. 03852) 

(f-1. 13673) *  (u-4. 03852) 
(d-0.17612) * (u-4. 03852) 
rt 

(ds-9. 77858) * (rt-199.098) 
(pwr-45. 9117) *  (rt-19  9.0  98) 

(f-1 .13673) *  (rt-19  9.0  98) 

(d-0.17  612) *  (rt-19  9.0  98) 

(u-4. 03852) *  (rt-19  9.0  98) 
log (rt ) 

(ds-9. 77 858) * (log (rt ) -4 . 70151 ) 
(pwr-45. 9117) * (log (rt) -4 . 70151) 
( f-1 .13673)* ( log (rt) -4.70151) 
(d-0.17612) * ( log (rt) -4.70151) 
(u-4. 03852) * ( log (rt) -4.70151) 
(rt-199.098) * (log (rt) -4 . 70151) 


Estimate 
-8.718660932 
-0.056903123 
0 .011512262 
0.001044546 
-0.505493111 
0.006345062 
0.003332478 
-1 . 120487986 
0.194697425 
0.000743776 
1.593317154 
0.011344108 
-0.001534965 
-0.000373762 
0.024318625 
-0 .104387249 
-0.039274128 
9.33325E-05 
-2 . 1 6774E-05 
0.00125172 
0 . 028577504 
0.000200101 
3.657781759 
-0.017403707 
0.00044408 
-0.039626276 
-7 . 679328346 
-0.050693625 
0 .013374708 


Table  A.5:  Area  3 


Term 

Estimate 

Intercept 

-5.855816349 

ds 

-0.043061243 

P 

0.026390456 

(ds-15 . 6968 ) * (p-46.0043) 

0.001218131 

f 

-0.732197524 

(ds-15. 6968) * (f-1. 10713) 

-0.018696361 

(p-46.0043) *  (f-1. 10713) 

0.017039565 

d 

-2.505611768 

(ds-15. 6968) * (d-0.0694) 

0.449824114 

(p-46.0043) * (d-0.0694) 

0.008519344 

(f-1. 10713) * (d-0.0694) 

-0.264137436 

u 

0.03056635 

(ds-15. 6968) * (u-4.1808) 

0 . 002540722 

(p-46.0043) *  (u-4.1808) 

-0.000298798 

(f-1. 10713) *  (u-4.1808) 

0.029593653 

(d-0.0694) * (u-4.1808) 

-0.021669586 

rt 

-0.031050663 

(ds-15. 6968) * (rt-193.92) 

3. 63672E-05 

(p-46.0043) *  (rt-193.92) 

-1 . 4 938 6E-05 

(f-1. 10713) *  (rt-193.92) 

0.000170988 

(d-0.0694) * (rt-193.92) 

0.076478561 

(u-4.1808) *  (rt-193.92) 

0.000173894 

log (rt ) 

2.592817848 

(ds-15. 6968) * (log(rt) -4 . 65234) 

0.001334408 

(p-46.0043) * (log(rt) -4 . 65234) 

0.000203602 

(f-1. 107 13) * (log(rt) -4 . 65234) 

0 . 11811913 

(d-0.0694) * (log(rt) -4 . 65234) 

-19.22459429 

(u-4.1808) * (log(rt) -4 . 65234) 

-0.051854286 

(rt-193.92) * (log(rt) -4 . 65234) 

0.010221632 

Table  A.6:  Area  4 


Term 

Intercept 

ds 

P 

(ds-15.603) * (p-45.9308) 
f 

(ds-15.603) * (f-1. 12139) 
(p-45.9308) *  (f-1. 12139) 
d 

(ds-15.603) * (d-0. 06935) 
(p-45.9308) * (d-0. 06935) 

(f-1. 12139) * (d-0. 06935) 
u 

(ds-15.603) * (u-4. 11415) 
(p-45.9308) *  (u-4. 11415) 

(f-1. 12139) *  (u-4. 11415) 

(d-0. 06935) * (u-4. 11415) 
rt 

(ds-15.603) * (rt-202 . 384 ) 

(p-4  5.9308) *  (rt-2  02. 384) 

(f-1. 12139) *  (rt-2  02. 384) 

(d-0. 06 935) * (rt-202. 384) 

(u-4. 114 15) *  (rt-2  02. 384) 
log (rt ) 

(ds-15.603) * ( log (rt) -4.73631) 
(p-45.9308) * ( log (rt) -4.73631) 

( f-1. 12 139) * ( log (rt) -4.73631) 
(d-0. 06 935) * ( log (rt) -4.73631) 
(u-4. 114 15) * ( log (rt) -4.73631) 
(rt-202. 384) * (log (rt) -4 . 73631) 


Estimate 
-14 . 62324806 
-0.064477288 
0 . 021111431 
0.001242567 
-0 .719757973 
-0 . 026085271 
0.012033069 
-4.978306236 
0.360854673 
-0.034237073 
1 . 818415375 
0.011592061 
-0.000413265 
0.000243691 
0.023445527 
0 . 074051142 
-0.047424143 
3. 93345E-05 
-3.377  99E-05 
0.001119479 
0.094306311 
0.000223639 
5.10630156 
0.002496818 
0.001259071 
0.023213712 
-23.59392878 
-0.057065023 
0.016656656 


This  remainder  of  this  appendix  considers  models  of  delay  for  the  four  areas.  First 


equations  (Two  way  parameter  interactions)  are  presented.  Then,  more  easily  read  tables. 


Table  A.  7  Area  1  (delay) 


Term 

Intercept 

ds 

P 

(ds-9. 74145) * (p-45.8749) 
f 

(ds-9. 74145) * (f-1. 12383) 
(p-45.8749) *  (f-1. 12383) 
d 

(ds-9. 74145) * (d-0.17633) 
(p-45.8749) * (d-0.17633) 

(f-1. 12383) * (d-0.17633) 
u 

(ds-9. 74145) * (u-4.09479) 
(p-45.8749) *  (u-4.09479) 

(f-1. 12383) *  (u-4.09479) 
(d-0.17633) * (u-4.09479) 
rt 

(ds-9. 74145) * (rt -199. 708) 
(p-45.874  9) *  (rt-19  9.7  08) 

( f-1. 12383 ) *  (rt-19  9.7  08) 
(d-0.17633) * (rt-199.708) 

(u-4.0  947  9) *  (rt-199.708) 
log (rt ) 

(ds-9. 74145) * ( log (rt) -4.71089) 
(p-45.8749) * ( log (rt) -4.71089) 
(f-1. 12383) * ( log (rt) -4.71089) 
(d-0.17633) * ( log (rt) -4.71089) 
(u-4.09479) * ( log (rt) -4.71089) 
(rt-199.708) * (log (rt) -4 . 71089) 


Estimate 

7.86836702 
0.064094049 
-0 . 001833424 
-9 . 8375E-05 
-0 . 028244187 
0 . 010371765 
0.009617641 
6.212019798 
0.060655473 
-0.010551809 
-0.038620384 
0.010069665 
0.003336211 
-0.000571154 
0.035373902 
0 . 154023307 
0.031169005 
-3. 61293E-05 
6 . 45355E-05 
-0.001864404 
-0.008542553 
3 . 33072E-05 
-3.389648479 
0.015538759 
-0.005680486 
0 . 034004137 
4.43820331 
0.012292536 
-0.010551076 


Table  A.8  Area  2  (delay) 


Term 

Intercept 

ds 

pwr 

(ds-9 . 77858 ) * (pwr-45 . 9117 ) 
f 

(ds-9. 77858) * (f-1. 13673) 
(pwr-45. 9117) *  (f-1. 1367  3) 
d 

(ds-9. 77 858) * (d-0.17612) 
(pwr-45. 9117) * (d-0.17612) 

(f-1 . 13673) * (d-0 . 17612) 
u 

(ds-9. 77858) * (u-4. 03852) 
(pwr-45. 9117) * (u-4. 03852) 

(f-1. 13673) *  (u-4. 03852) 
(d-0.17612) * (u-4. 03852) 
rt 

(ds-9. 77858) * (rt-199.098) 
(pwr-45. 9117) *  (rt-19  9.0  98) 

(f-1 .13673) *  (rt-19  9.0  98) 

(d-0.17  612) *  (rt-19  9.0  98) 

(u-4. 03852) *  (rt-19  9.0  98) 
log (rt ) 

(ds-9. 77 858) * (log (rt ) -4 . 70151 ) 
(pwr-45. 9117) * (log (rt) -4 . 70151) 
( f-1 .13673)* ( log (rt) -4.70151) 
(d-0.17612) * ( log (rt) -4.70151) 
(u-4. 03852) * ( log (rt) -4.70151) 
(rt-199.098) * (log (rt) -4 . 70151) 


Estimate 

9.447350375 
0.021980792 
0.00165042 
-0.00026741 
-0 .417104768 
-0.001437661 
-0.000127332 
5.807506024 
0.020307046 
0.013365543 
-1.413568285 
-0.002370652 
0.000399796 
-8 . 73275E-05 
0.00780889 
0.052039242 
0.031877858 
-2 . 01532E-05 
2 . 66785E-05 
-0.002102359 
-0.017390916 
3 . 38 665E-05 
-3.544955405 
0.008138538 
-0.000118306 
-0 . 114310292 
5.062926232 
0 .004863214 
-0 .011557246 


Table  A.9  Area  3  (delay) 


Term 

Estimate 

Intercept 

2.810766594 

ds 

0.024061631 

P 

-0 . 000376741 

(ds-15 . 6968 ) * (p-46.0043) 

-0.000233977 

f 

-0.244775334 

(ds-15. 6968) * (f-1. 10713) 

-0.002667492 

(p-46.0043) *  (f-1. 10713) 

-0.00113474 

d 

16.66138395 

(ds-15. 6968) * (d-0.0694) 

0.019501425 

(p-46.0043) * (d-0.0694) 

0.005022948 

(f-1. 10713) * (d-0.0694) 

-4.226402786 

u 

-0.008612198 

(ds-15. 6968) * (u-4.1808) 

-0.000612673 

(p-46.0043) *  (u-4.1808) 

-0.000334472 

(f-1. 10713) *  (u-4.1808) 

0.006607612 

(d-0.0694) * (u-4.1808) 

0.479712655 

rt 

0.018894808 

(ds-15. 6968) * (rt-193.92) 

-1 . 78761E-05 

(p-46.0043) *  (rt-193.92) 

4 . 5 9345E-05 

(f-1. 10713) *  (rt-193.92) 

-0.002494839 

(d-0.0694) * (rt-193.92) 

0 . 029154407 

(u-4.1808) *  (rt-193.92) 

3 . 53694E-05 

log (rt ) 

-1 . 939976307 

(ds-15. 6968) * (log(rt) -4 . 65234) 

0.004324752 

(p-46.0043) * (log(rt) -4 . 65234) 

-0.002663753 

(f-1. 107 13) * (log(rt) -4 . 65234) 

0.028002515 

(d-0.0694) * (log(rt) -4 . 65234) 

4 . 698102582 

(u-4.1808) * (log(rt) -4 . 65234) 

0.006634126 

(rt-193.92) * (log(rt) -4 . 65234) 

-0.006094104 

Table  A.10  Area  4  (delay) 


Term 


Estimate 


Intercept 

10 

ds 

0. 

P 

1 . 

(ds-15.603) 

*  (p-45.9308) 

-0 

f 

-0. 

(ds-15.603) 

*  (f-1 . 12139) 

0. 

(p-45.9308) 

*  (f-1. 12139) 

0. 

d 

15 

(ds-15.603) 

* (d-0. 06935) 

0. 

(p-45.9308) 

* (d-0. 06935) 

-0. 

(f-1 . 12139) 

* (d-0. 06935) 

-2 

u 

0. 

(ds-15.603) 

* (u-4 . 11415) 

0. 

(p-45.9308) 

* (u-4. 11415) 

-5. 

(f-1. 12139) 

* (u-4. 11415) 

0. 

(d-0. 06935) 

* (u-4 . 11415) 

0. 

rt 

0. 

(ds-15.603) 

* (rt-202.384) 

-2  . 

(p-45.9308) 

* (rt-202.384) 

5. 

(f-1. 12139) 

* (rt-202.384) 

-0 

(d-0. 06935) 

* (rt-202.384) 

-0. 

(u-4. 11415) 

* (rt-202.384) 

7  . 

log (rt ) 

-3. 

(ds-15.603) 

* (log(rt)-4. 73631) 

0. 

(p-45.9308) 

* (log(rt)-4. 73631) 

-0. 

(f-1 . 12139) 

* (log(rt)-4. 73631) 

-0. 

(d-0. 06935) 

* (log(rt)-4. 73631) 

12 

(u-4. 11415) 

* (log(rt) -4 .73631) 

0. 

(rt-202.384) * (log (rt) -4 . 73631) 

-0. 

.48802137 
025672549 
5  93  63E-05 
.00028144 
239425078 
010371208 
004645574 
.32668466 
146743028 
006239019 
.  12234659 
011929188 
001463114 
55474E-05 
024330278 
021202838 
034251733 
83939E-05 
57  67  3E-05 
.00199673 
034404391 
17  3  62E-05 
953824206 
004869515 
002830166 
061591662 
.  66271853 
005611116 
011938993 
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Motivation:  Address  “Value  of  Pound 

of  C4ISR”  Question 


1 .  Need  to  capture  marginal  impact  of  technology  options 

How  much  better  is  C4ISR  performance  given: 

•  Additional  bandwidth,  new  technologies  (e.g.  radios,  antennas,  etc.) 

•  More  (or  less)  sensor  data 

•  More  (or  less)  frequent  COP  update 

•  Enhanced  connectivity 

2.  Need  to  quantify  C4ISR  benefits  into  combat  outcomes 


What  are  effects  on  warfighter? 
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Source:  Fisher,  2003 


PLATFORMS  &  SENSORS 


Page 


Approach:  Capture  Technology  and  Scenario- 

Specific  Detail 


•  Network  performance  highly  sensitive  to  technology  detail 
and  scenario  specifics 

•  Technology  options  and  combinations  of  options  are  numerous 

•  Terrain/scenario  has  a  large  impact 

•  Vehicle  characteristics  (e.g.,  mobility)  affects  network  performance, 
reliability,  etc. 


J ANUS/ JC ATS/ 
Other 

Factors  in  details  of 
operations 
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Hi-resolution 

Combat 

Simulation 

Tool 


Hi-resolution 
Communication 
Network 
Simulation 

Tool  /  j  Qualnet 

Factors  in  details 

of  technologies 


Qualnet  Simulations  Used  To  Develop 

Performance  Curves 


Parameters 

(Tech, 

Options) 

•Traffic  volume  &  type 
•Terrain  type 
•Number  of  Nodes 
•Node  mobility 
•Other  factors 


Qualnet  Simulations 


Performance  Data 


Warfighter 

Effectiveness^ 

e.g.,  mission 
execution, 
attrition,  etc. 
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JANUS  Simulations 
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Why  a  Meta-Model? 


1.  Communication  network  simulation  is 
complex  and  time  consuming 

2.  Meta-models  allow  flexibility  while  not  adding 
large  overhead  time  to  combat  simulations 


3.  Regression  analysis  can  be  used  to  generate 
model  “off-line” 


Terrain/Scenario  Being  Studied 


inside  individual  boxes  is 
modeled 

Boxes  vary  in  size  and 
terrain  roughness 


RAND 


Area  3 
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Example  Of  Simulation  Experiments  (Area  #1) 
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Experiment: 

•  Bn-light  dispersed 
across  Terrain 

•  Data  multicasted  out 
to  nodes  at  varying 
rate 


•  Performance 

captured  as  function 
of  frequency, 
mobility,  UAV 
usage,  etc. 


Data  out  to  all 
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Factors  of  Interest  and  Responses 


Factors 

Terrain, 

Radio  Specs, 
etc. 

Frequency 

Radio  Power 

Network  Density 

Network  Simulator 

(Qualnet) 

w 

#  UAVs 

w 

Radio  throughput  rate 

w 

Data  rate 

w 

Distance  between  nodes 

w 

Line  of  Sight  between  node^s 

Responses 


Packet  Delivery  ratios 


Delay  (seconds) 


RAND 
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Design  of  Experiments:  Simulations  Run  at  Various 

Levels  of  Each  Factor 


Design  Matrix 


Sim  run 

freq 

#  UAVs 

Pwr 

Density 

(nodes) 

Radio 

Cap 

Ant. 

Hgt 

Deli 

very 

Delay 

1 

2GHz 

0 

20W 

145 

6Mbps 

2.5m 

% 

sec 

2 

1  GHz 

8 

20W 

72 

6Mbps 

5  m 

3 

.4  GHz 

4 

20W 

36 

2Mbps 

10  m 

4 

2  GHz 

0 

20W 

145 

2Mbps 

2  m 

5 

1GHz 

8 

20W 

72 

2Mbps 

5  m 

1000-3000  Experiments 
Run  for  each  Area 


Closed  Form  Expression  Developed  to 
Capture  Performance  as  Function  of 

Demand  for  UA  level 


Logit  (pdr)  = 


(3  j  (F  requency)+P2(UA  V  s)+ . 


Other 

•  •-<;  First-Order 


Terms 


P3(FrequencyxUAVs)+. . . 


Other 

A  Second-Order 
Terms 


Other  Higher  Order  Interactions 
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Evaluating  The  Fit  For  One  Measure 


Meta-model  predicted  PDR  Meta-model  predicted  PDR 


RAND 


Logit(p)=f(frequency,  #  UAVs,  density,  data  traffic,  distance..) 

exp(log  #(/?)) 

(exp(log/Y(  p))  + 1 
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Evaluating  The  Fit  (Cont.) 


Table:  Evaluating  the  fits  for  2-way  parameter  interaction 


Area/ 

Equation 

Adjusted  R- 
Square 
Value  for 

PDR 

Adjusted  R- 
Square  Value 
for  Delay 

1 

0.715 

0.755 

2 

0.749 

0.817 

3 

0.725 

0.813 

4 

0.706 

0.800 

Note:  The  fits  can  be  improved  by  representing  more  than  two-way  parameter  interaction  in  the  model. 


RAND 
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Evaluating  The  Fit  (Cont.) 


Table:  Evaluating  the  Fits  for  4-way  Parameter  Interaction 


Area 

Equation 

Ad j .  R- 
Square 
Value  for 

PDR 

Ad j .  R- 
Square 
Value  for 
Delay 

Ad j .  R- 
Square 
Value 
for  PDR, 

T  ,05 

Ad j .  R- 
Square 
Value  for 
Delay, 

LOS 

1 

0 . 72 

0.78 

2 

0.75 

0.84 

0 . 77 

0 . 85 

3 

0 . 73 

0.84 

0.78 

0 .86 

4 

0.71 

0.82 

— 

— 

RAND 
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Some  Analysis  Results 
Using  The  Metamodels 


RAND 
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Benefit  of  UA  Vs  Depends  on  Density  of 

Forces 


Area  3  Performance 


Area  2  Performance 


Nodes  in  40km  x  40  km  Area 


0  UAVs 
4  UAVs 
8  UAVs 


Density 


Density 
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Results  are  Robust  Across  Terrain 


Example  Analysis  Facilitated  by  M&S:  Impact 

of  UAVs  Quantified 
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Plot  of  Packet  Delivery  Ratio  Between  Nodes 
as  a  Function  of  Line-of-Sight  and  Distance 
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UAVs  Improve  Performance  Across  Data  Rates 
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Higher  Frequency  (2.5  GHz), 

Medium  Data  Rate  Radios:  UAVs  are  Critical 
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But  Greater  Gains  Achieved  At  Better  Propagating 

Frequencies 

Low  Frequency  (.4GHz) 

Medium  Data  Rate  Radios,  UAVs  arent  critical 
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-B—  8  UAVS 


100 


200 


300 


400 


500 


600 


Data  Rate  in  Kbps 


“Better”Channel  allocation  Requires  Fewer  UAVs  to  Maintain  High  performance 
RAND 
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Better  still:  High  Radios  (6  Mbps)  JTRS 
Radios  Provide  Big  Performance 

Enhancement 


For  example: 


High  Throughput  Radio  Performance  at  10 
kilometers  (Area  1) 


—  w/UAVs 

—  w/oUAVs 


2.5  GHz  Tx 


RAND 


145  Node  network  in  25km  x  25km  area 
with  6Mbps  Radios 


Model  of  Area  1  Used  Above 


Observations  from  Modeling  Effort 


*  High  bandwidth  tactical  radios  will  help  (>  5  Mbps 
user  throughput) 

*  Near  future  radios  (1-2  Mbps  user  throughput)  will 
require  significant  UAV  presence  to  ensure  reliable 
C2/SA  network 

-  Depending  on  force  size 

-  Information  dissemination  requirements 

-  Spectrum  to  support  them  a  big  issue 

*  Frequency  agile,  cognitive  radios  have  potential  to  be 
advantageous 

RAND 
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Next  Step:  QUALNET  Derived  Data  Communication 
Model  Inserted  in  Combat  Simulator 


LOW  UAV  DENSITY 


U  of  A 
#1 


RAND 


LOW 

TRANSMISSION 

ACTIVITY 


MEDIUM 

TRANSMISSION 

ACTIVITY 


HIGH 

TRANSMISSION 

ACTIVITY 
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Back 


RAND 


ups 
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Line  of  Sight  (LOS)  Was  Useful  Factor 


RAND 
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Message  Completion  Rate  or 
Likelihood  Message  gets  Through  (%) 


Why  Are  Packet  Requirements  So  Critical? 

Likelihood  COP  Update  Received 

100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20% 

10% 

0% 

0  100  200  300  400  500  600  700 

Size  of  COP  Update  (Kilobits) 


RAND 


Message  Completion  Rates  Highly  Sensitive 
to  Packet  Delivery  Ratios 
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Sample  of  Results  From  Experiments 


Area 

#  of  UAVs 

Density 

Nodes/km2 

Frequency 

2  (25  x25) 

8 

.12 

2.5  GHz 

2  (25  x25) 

4 

.12 

2.5  GHz 

2  (25  x25) 

0 

.12 

2.5  GHz 

2  (25  x25) 

8 

.12 

0.4  GHz 

2  (25  x25) 

4 

.12 

0.4  GHz 

2  (25  x25) 

0 

.12 

0.4  GHz 

2  (25  x25) 

8 

.06 

2.5  GHz 

2  (25  x25) 

4 

.06 

2.5  GHz 

2  (25  x25) 

0 

.06 

2.5  GHz 

90%  PDR 
Data  Rate 

270  Kbps 
255  Kbps 
240  Kbps 
320  Kbps 
345  Kbps 
375  Kbps 
110  Kbps 
40  Kbps 
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